OME-Zarr copies: asked on the first screen, written as a recorded conversion once the axes are confirmed (redesign 3, part 4) - #287

Local gate (python tools/check.py: ruff check, ruff format --check, pytest, code rules) on the final head of every branch of the stack, each with main a775ebf (#288) merged:

The 4 skips are the same on every head and are environment skips: test_cli_log.py:166, test_folder_rules.py:199 and :230 (this file system tells names apart by case), test_discovery.py:665 (running with permissions that ignore file modes).

Same answers, main a775ebf against 351c134, on 28 images of 2048 × 2048 with 3 channels, masks, pixel sizes differing in the 5th decimal, one size stated for a folder and one for all: morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json, mask_representation.json are byte-identical; run_manifest.json is identical apart from times and the two output paths. tests/spec_fixtures and the Spec tests are untouched.

Changes made for CI only, in the test helpers: the kernel runs of the notebook file are repeated when the client loses the kernel's output (macOS runners, "Timeout waiting for IOPub output"), and two path assertions compare the folder as the record spells it (Windows).

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

Generated by Claude Code

Final heads (this replaces the table above; main a775ebf, #288, is merged into each). python tools/check.py exit 0 on every head; CI 7/7 green on every head.

The 4 skips are environment skips, the same on every head (case-sensitive file system ×3, file modes ×1).

The macOS failures of the earlier heads, explained. The app's cell sends one IOPub message per widget model: 1,837 on main, 2,015 with #284 and 2,135 at the top of the stack (counted with nbclient). ZeroMQ keeps 1,000 messages on each side for a subscriber and drops the rest, so above 2,000 a client that reads slowly (nbclient on the macOS runners) lost the end of the stream, with the cell's display message. tests/test_notebook_file.py::_execute now gives the kernel client an unlimited receive queue (and repeats a run that still loses its output, twice at most). JupyterLab and Colab read the stream as it comes; this was seen only with nbclient in CI. Reducing the number of widget models the app creates at start (lazy pages) would remove the cause and is not done here.

Byte comparison (same answers, main a775ebf against the top): morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json, mask_representation.json identical; run_manifest.json identical apart from times and the two output paths.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

Generated by Claude Code

Independent review: #287 (OME-Zarr copy option)

Verdict: approve after one fix (pushed), with owner questions. Merged the projection branch forward (e856f6f, then 5542523 with the #286 fix), with no conflicts. Retargeted to main after #286 merged (be5cead).

Defect found and fixed: 1f5dc17. The copies were not OME-NGFF 0.4 valid. I validated .zattrs against the official https://ngff.openmicroscopy.org/0.4/schemas/image.schema (jsonschema, draft 2020-12). Every copy with channel names failed: 'window' is a required property, 'color' is a required property on each omero.channels[] item. Each channel now carries color: "FFFFFF" and a window equal to the pixel type's whole range (min, max, start, end). Both are display only and never read from the pixels, so one source still gives the same bytes. The brick's reader is unchanged. Contract §17 and the changelog say so. Regression test: test_the_omero_channels_have_what_ome_ngff_0_4_requires. It fails without the fix.

After the fix my script wrote YX, CYX and TCYX (named) and TCZYX/ZCYX (max, mean, sum projections; uint16 and float32). All of them:

• validate against image.schema 0.4;
• have axes in t, c, then space order, with 2 or 3 space axes;
• have NGFF units (micrometer, second), dimension_separator /, and unknown_scale for a missing size;
• have pixels equal bit for bit to the source read in t,c,z,y,x order, and projections equal to numpy's (mean within 1e-9);
• give byte-identical stores on a second write;
• leave the originals' SHA-256 unchanged.

The record stays "3" with source_relative_path. (strict_image.schema additionally wants the SHOULD fields name, type and metadata on the multiscale. They are not written. This is informational, not invalid.)

Also checked:

• Ticking the option writes nothing. Nothing is written before Confirm these axes, and the confirmation itself writes nothing. The card is disabled with "the axis roles are not confirmed" until then. After the copy, the copies' axes are not confirmed for the user.

Owner questions. My recommendation comes first in each.

1. Z-stacks + OME-Zarr write before Confirm axis roles. With the option ticked, the projection card writes projected_<method>_zarr on the example tick plus the method plus the click. Stacks can never be "confirmed" (Z>1 is refused), so this is the same statement of roles as the OME-TIFF projection. The PR body discloses it. Recommendation: keep it. The alternative is to write the projections as OME-TIFF and offer the OME-Zarr copy after the copies' axes are confirmed, which makes a copy of a copy.
2. The option stays ticked when the folder changes. It is a first-screen preference, not an answer about the data: nothing is written without the Axes card's click, which names the destination. Recommendation: keep it. Resetting it would also untick it when the app opens the copies.
3. Record version. source_relative_path is optional at v3. A v0.5.1 reader would ignore it and report a false conversion_source_changed warning, not an error. Recommendation: raise to v4 if strict refusal is the rule (AR10 already refuses other versions). Not changed here.

Byte identity (re-measured on 1f5dc17 against main a685711, same input files, same answers through the stepper):

• Run with masks (Nuclei instances, Mito grouped labels, plus the mask step): morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json and mask_representation.json are byte-identical. run_manifest.json is identical apart from times and paths.
• Run with 3 channels in rep1 and 4 in rep2 (Stage 3 S3.3: per-unit channel lists, plane_histograms v5 #288 Stage 3): same result.

The only other differences are in discovery_report.*, and they are the manifest's own SHA-256, which follows from its times.

Gates (Ruff check, Ruff format, code rules, pytest -n 2; the skips are the same four everywhere: root-user file modes (one), case-sensitive file system (three)):

CI on 1f5dc17 is 14/14 green.

Generated by Claude Code